MPCViT: Searching for MPC-friendly Vision Transformer with Heterogeneous Attention

Secure multi-party computation (MPC) enables computation directly on encrypted data on non-colluding untrusted servers and protects both data and model privacy in deep learning inference. However, existing neural network (NN) architectures, including Vision Transformers (ViTs), are not designed or optimized for MPC protocols and incur significant latency overhead due to the Softmax function in the multi-head attention (MHA). In this paper, we propose an MPC-friendly ViT, dubbed MPCViT, to enable accurate yet efficient ViT inference in MPC. We systematically compare different attention variants in MPC and propose a heterogeneous attention search space, which combines the high-accuracy and MPC-efficient attentions with diverse structure granularities. We further propose a simple yet effective differentiable neural architecture search (NAS) algorithm for fast ViT optimization. MPCViT significantly outperforms prior-art ViT variants in MPC. With the proposed NAS algorithm, our extensive experiments demonstrate that MPCViT achieves 7.9x and 2.8x latency reduction with better accuracy compared to Linformer and MPCFormer on the Tiny-ImageNet dataset, respectively. Further, with proper knowledge distillation (KD), MPCViT even achieves 1.9% better accuracy compared to the baseline ViT with 9.9x latency reduction on the Tiny-ImageNet dataset.Comment: 6 pages, 6 figure

Predicting Continuous Locomotion Modes via Multidimensional Feature Learning from sEMG

Walking-assistive devices require adaptive control methods to ensure smooth transitions between various modes of locomotion. For this purpose, detecting human locomotion modes (e.g., level walking or stair ascent) in advance is crucial for improving the intelligence and transparency of such robotic systems. This study proposes Deep-STF, a unified end-to-end deep learning model designed for integrated feature extraction in spatial, temporal, and frequency dimensions from surface electromyography (sEMG) signals. Our model enables accurate and robust continuous prediction of nine locomotion modes and 15 transitions at varying prediction time intervals, ranging from 100 to 500 ms. In addition, we introduced the concept of 'stable prediction time' as a distinct metric to quantify prediction efficiency. This term refers to the duration during which consistent and accurate predictions of mode transitions are made, measured from the time of the fifth correct prediction to the occurrence of the critical event leading to the task transition. This distinction between stable prediction time and prediction time is vital as it underscores our focus on the precision and reliability of mode transition predictions. Experimental results showcased Deep-STP's cutting-edge prediction performance across diverse locomotion modes and transitions, relying solely on sEMG data. When forecasting 100 ms ahead, Deep-STF surpassed CNN and other machine learning techniques, achieving an outstanding average prediction accuracy of 96.48%. Even with an extended 500 ms prediction horizon, accuracy only marginally decreased to 93.00%. The averaged stable prediction times for detecting next upcoming transitions spanned from 28.15 to 372.21 ms across the 100-500 ms time advances.Comment: 10 pages,7 figure

Gait Cycle-Inspired Learning Strategy for Continuous Prediction of Knee Joint Trajectory from sEMG

Predicting lower limb motion intent is vital for controlling exoskeleton robots and prosthetic limbs. Surface electromyography (sEMG) attracts increasing attention in recent years as it enables ahead-of-time prediction of motion intentions before actual movement. However, the estimation performance of human joint trajectory remains a challenging problem due to the inter- and intra-subject variations. The former is related to physiological differences (such as height and weight) and preferred walking patterns of individuals, while the latter is mainly caused by irregular and gait-irrelevant muscle activity. This paper proposes a model integrating two gait cycle-inspired learning strategies to mitigate the challenge for predicting human knee joint trajectory. The first strategy is to decouple knee joint angles into motion patterns and amplitudes former exhibit low variability while latter show high variability among individuals. By learning through separate network entities, the model manages to capture both the common and personalized gait features. In the second, muscle principal activation masks are extracted from gait cycles in a prolonged walk. These masks are used to filter out components unrelated to walking from raw sEMG and provide auxiliary guidance to capture more gait-related features. Experimental results indicate that our model could predict knee angles with the average root mean square error (RMSE) of 3.03(0.49) degrees and 50ms ahead of time. To our knowledge this is the best performance in relevant literatures that has been reported, with reduced RMSE by at least 9.5%

A Versatile Surface Bioengineering Strategy Based on Mussel-Inspired and Bioclickable Peptide Mimic

In this work, we present a versatile surface engineering strategy by the combination of mussel adhesive peptide mimicking and bioorthogonal click chemistry. The main idea reflected in this work derived from a novel mussel-inspired peptide mimic with a bioclickable azide group (i.e., DOPA4-azide). Similar to the adhesion mechanism of the mussel foot protein (i.e., covalent/noncovalent comediated surface adhesion), the bioinspired and bioclickable peptide mimic DOPA4-azide enables stable binding on a broad range of materials, such as metallic, inorganic, and organic polymer substrates. In addition to the material universality, the azide residues of DOPA4-azide are also capable of a specific conjugation of dibenzylcyclooctyne- (DBCO-) modified bioactive ligands through bioorthogonal click reaction in a second step. To demonstrate the applicability of this strategy for diversified biofunctionalization, we bioorthogonally conjugated several typical bioactive molecules with DBCO functionalization on different substrates to fabricate functional surfaces which fulfil essential requirements of biomedically used implants. For instance, antibiofouling, antibacterial, and antithrombogenic properties could be easily applied to the relevant biomaterial surfaces, by grafting antifouling polymer, antibacterial peptide, and NO-generating catalyst, respectively. Overall, the novel surface bioengineering strategy has shown broad applicability for both the types of substrate materials and the expected biofunctionalities. Conceivably, the “clean” molecular modification of bioorthogonal chemistry and the universality of mussel-inspired surface adhesion may synergically provide a versatile surface bioengineering strategy for a wide range of biomedical materials

Impact of Nano–Sized Polyethylene Terephthalate on Microalgal–Bacterial Granular Sludge in Non–Aerated Wastewater Treatment

The widely used plastics in our daily lives have resulted in ubiquitous microplastics and nanoplastics in wastewater, such as polyethylene terephthalate (PET). As an emerging green process for wastewater treatment and resource recovery, microalgal–bacterial granular sludge (MBGS) aligns with the concept of the circular economy. However, it is unclear whether the tiny PET can affect the MBGS process. Thus, this study investigated the impact of nano–sized PET (nPET) on the MBGS process. The results showed that 10 to 30 mg/L nPET had no obvious impact on pollutant removal as compared with the control group. However, the performance of the MBGS with the addition of 50 mg/L nPET became worse after 15 days. Scanning electron microscopy revealed that the MBGS adsorbed nPET by generating extracellular polymeric substances. Further microbial analyses showed that the algal abundance in prokaryotes slowly declined with increasing concentrations of nPET, while the reduced energy storage and electron transfer in eukaryotes might lead to an inferior performance at 50 mg/L nPET. Overall, the MBGS was demonstrated to exhibit good adaptability to nPET–containing wastewater, which showed the potential to be applied for the treatment of municipal wastewater containing nanoplastics

Impact of Environmental Factors on the Formation and Development of Biological Soil Crusts in Lime Concrete Materials of Building Facades

Microbial colonization leads to the formation of biological soil crusts (BSCs) on the surface of architecture, which causes the deterioration of construction materials. However, little information is available on the formation of BSCs on lime concrete materials of early architecture. In this study, the variances of microbial communities, physicochemical properties, and surrounding environmental factors of the lime concrete facades from the early architecture of Wuhan University were investigated. It was found that the surface of lime concrete materials was internally porous and permeable, embedded with biofilms of cyanobacteria, mosses, bacteria, and fungi. Redundancy analysis (RDA) analysis showed that the abundances of photoautotrophic microorganisms depended on light intensity and moisture content of construction materials, while that of heterotrophic microorganisms depended on total nitrogen (TN) and NO3−-N content. The deposition of total carbon (TC), NH4+-N, and total organic carbon (TOC) was mainly generated by photoautotrophic microorganisms. The lime concrete surface of early architecture allowed internal growth of microorganisms and excretion of metabolites, which promoted the biodeterioration of lime concrete materials

Impact of Environmental Factors on the Formation and Development of Biological Soil Crusts in Lime Concrete Materials of Building Facades

Microbial colonization leads to the formation of biological soil crusts (BSCs) on the surface of architecture, which causes the deterioration of construction materials. However, little information is available on the formation of BSCs on lime concrete materials of early architecture. In this study, the variances of microbial communities, physicochemical properties, and surrounding environmental factors of the lime concrete facades from the early architecture of Wuhan University were investigated. It was found that the surface of lime concrete materials was internally porous and permeable, embedded with biofilms of cyanobacteria, mosses, bacteria, and fungi. Redundancy analysis (RDA) analysis showed that the abundances of photoautotrophic microorganisms depended on light intensity and moisture content of construction materials, while that of heterotrophic microorganisms depended on total nitrogen (TN) and NO3−-N content. The deposition of total carbon (TC), NH4+-N, and total organic carbon (TOC) was mainly generated by photoautotrophic microorganisms. The lime concrete surface of early architecture allowed internal growth of microorganisms and excretion of metabolites, which promoted the biodeterioration of lime concrete materials